Technologies for digitization and compression of picture and audio data which are originally analog have made progress. The merits of using digitized data are as follows: various types of data including picture, audio, and text data can he treated uniformly; application of compression technology in recording or communicating data can utilize the capacity of storage unit and limited transmission bandwidth fully and effectively while keeping a quality of data to be recorded or transmitted; and high technologies for error correction and encryption can be used with ease.
For transmitting and receiving the digitized data and the compression-coded data, packetizing is generally performed. Packetizing means generating packets by splitting data into pieces of a given size and adding information to them, and transmitting and receiving data as packets can improve efficiency and precision of data communication. In packet switching in a computer network, for example, packets are transmitted separately and at different timings, through the network to their destination where they are reassembled to original data. Therefore, to each packet, information indicating a destination, a transmission source, and an order is added to each packet.
In transmission of packetized digital data, for example, plural types of digital data including picture, audio, and text information are split into pieces, respectively, and the information for utilizing the digital data is added to each piece to packetize the data. Then, multiplexing processing is performed on the assumption that packets based on plural types of data constitute a stream, and resultant multiplexed data is transmitted.
As an international standard method for technologies of compression-coding digital data such as picture, audio, and text data, multiplexing plural pieces of compression-coded digital data, and transmitting the resultant data, Moving Picture Experts Group, Phase 2 (MPEG2) has become pervasive (for reference purpose, see ISO/IEC JTC1/SC29/WG11 N801,"ISO/IEC 13818 1 International Standard: INFORMATION TECHNOLOGY-GENERIC CODING OF MOVING PICTURES AND ASSOCIATED AUDIO: SYSTEMS", 1994, November), and Moving Picture Exports Group, Phase. 4 (MPEG4) is being under study (for reference purpose, see ISO/IEC JTC1/SC29/WG11 N1483, "Systems Working Draft Version 2.0", 1996, November). MPEG4 has a feature that digital data including picture, audio, and text data are treated, and in each case, specific components are treated as objects; for example, the objects included in a picture are used as objects. FIG. 8 is a conceptual diagram for explaining object processing. Hereinafter a description is given of the object processing with reference to FIG. 8.
Digital picture data obtained by digitizing a picture is identical to a sequence of still picture data, and one piece of the picture data corresponding to one screen (frame) is called a frame picture. As shown in FIG. 8, it can be thought that one frame picture S801 is composed of three objects: a background S802, a water plant S803, and a fish S804. In the object processing for moving picture, the processing efficiency is improved generally by treating specific objects included in a moving picture (the water plant and the fish in this case) and the other objects (a background in this case) separately.
In MPEG4, each object data is compression-coded and multiplexed with time information that consists of a time clock reference and presentation time information of each object data, and the resultant multiplexed data is transmitted or recorded. Time information is used for appropriate decoding of the multiplexed data transmitted or recorded to obtain pictures.
FIG. 9 is a block diagram illustrating the construction of a prior art decoding apparatus whose processing object is such multiplexed data. As shown in the figure, the prior art decoding apparatus comprises a demultiplexing means 901, a decoding buffer 902, an object decoding means 903, a composition buffer 904, a compositing means 905, a composited data buffer 906, a presenting means 907, and a time clock generating means 908.
The demultiplexing means 901 receives multiplexed data which is a processing object for the decoding apparatus, as an input, and demultiplexes the same to generate a predetermined number of pieces of coded object data, time clock reference information, and presentation time information of object data. In this case, three pieces of object data are used.
The decoding buffer 902 temporarily stores the coded object data which are processing objects of decoding processing. The decoding buffer 902 comprises first to third decoding buffers 902a to 902c used for temporarily storing the first to third coded object data. The object decoding means 903 performs decoding processing to the coded object data temporarily stored in the decoding buffer to generate decoded object data. The object decoding means 903 comprises first to third object decoding means 903a to 903c used for processing the first to third object data. The composition buffer 904 temporarily stores the decoded object data which is a processing object of composition processing, and comprises first to third composition buffers 904a to 904c employed for temporary storage of the first to third decoded object data.
The compositing means 905 composites pieces of decoded object data to generate composited object data. The composited data buffer 906 temporarily stores composited object data which is an object to be presented. The presenting means 907 presents composited object data based on presentation time information of object and a time clock which will be described later. The time clock generating means 908 generates a time clock used for decoding processing, based on the time clock reference information which has been multiplexed with the coded data to be transmitted.
Hereinafter a description is given of the operation for processing multiplexed data in the prior art decoding apparatus which is thus constructed.
First of all, multiplexed data S951 which is a processing object for the decoding apparatus is input to the demultiplexing means 901. The demultiplexing means 901 demultiplexes the multiplexed data S951 to generate first to third coded object data S952, S953, and S954, time clock reference information S955, and presentation time information of object data S956. Then, the demultiplexing means 901 outputs the first to third coded object data S952 to S954 to the first to third decoding buffers 902a to 902c, the time clock reference information S955 to the time clock generating means 908, and the presentation time information S956 to the presenting means 907, respectively. The first to third decoding buffers 902a to 902c store the first to third input coded object data S952 to S954 until the first to third object decoding means 903a to 903c fetch them, respectively.
The first to third object decoding means 903a to 903c fetch the coded object data S952 to S954 from the first to third decoding buffers 902a to 902c, respectively, and perform decoding processing to generate the first to third decoded object data S957 to S959, outputting the same to the first to third composition buffers 904a to 904c, respectively. The first to third composition buffers 904a to 904c store the first to third decoded object data S957 to S959 until the compositing means 905 fetches the same.
The compositing means 905 fetches the first to third decoded object data S957 to S959 from the first to third composition buffer 904a to 904c, respectively, and performs composition processing to generate composited object data S960, outputting it to the composited data buffer 906. The composited data buffer 906 stores the input composited object data S960 until the presenting means 907 fetches the same.
On the other hand, the time clock generating means 908, which has a function of generating clock signals, generates time clock information S961 based on time clock reference information S955 using the generated clock signals, and outputs it to the presenting means 907. The time clock information S961 is the time information used for the processing in the decoding apparatus. The presenting means 907 compares the input time clock information S961 with the presentation time information S956. When it is judged that the time clock information S961 conforms to the presentation time information S956, this presenting means fetches the composited object data S960 stored in the composited data buffer 906 to present it.
As described above, in the prior art picture decoding apparatus, use of decoding buffer, composition buffer, and composited data buffer enables to perform demultiplexing of multiplexed data, decoding processing, composition processing, and presentation simultaneously while receiving multiplexed data. As these buffers, employed are relatively high-speed storage media such as semiconductor memories. Because the buffers' capacity is finite, in the prior art decoding apparatus, the following controls are performed to he respective buffers included therein.
In the decoding buffer 902a, when object data "Data 1" is stored, a storage area of the decoding buffer (storage medium) where "Data 1" is stored is protected from writing. Therefore, "Data 2", which is the data input following "Data 1", is stored in any storage area of the decoding buffer 902a other than the area where "Data 1" is stored. Consequently, "Data 1" is held in the decoding buffer 902a until the decoding processing at the subsequent stage is performed, protected from being overwritten by the subsequent data to be stored. Then, when "Data 1" is subjected to processing by the object decoding means, the protection over the storage area of the decoding buffer 902a where "Data 1" is stored is canceled. This means that it is possible to overwrite the subsequent data, "Data N", in the area where "Data 1" is stored.
Also in the buffers other than the decoding buffer 902a, the same buffer control is performed. In each buffer, when the data stored therein is subjected to the processing at the subsequent stage, namely, composition processing for the composition buffer and presentation for the composited data buffer, overwriting can be performed in the storage area which the data has occupied.
As described above, in the prior art decoding apparatus, by means of buffers which are storage media of finite capacities, multiplexed data is demultiplexed to obtain coded object data and time information, and the coded object data is decoded, composited, and presented correctly according to time information.
In the prior art decoding apparatus, the abovedescribed control over data storage is performed in each of the decoding buffer 902, the composition buffer 904, and the composited data buffer 906, and input data is held until a processing is performed at a subsequent stage, but the data stored in each buffer cannot be treated as a processing object of the same processing two times.
FIG. 10 is a diagram for explaining a possibility of reusing object data in the decoding apparatus. As shown in the figure, frame pictures S1001 to S1004 are picture data ordered in a time series, and in the decoding apparatus, they are presented in this order. In the case of treating a fish included in each frame picture as specific object data, the object data of the fish included in the frame picture S1001 is identical to that included in the frame picture S1003, and reuse of the data is possible as long as the data is held. The reuse of data can reduce consumed capacity of recording media and occupancy of transmission paths, as well as amount of coded object data.
Further, in games treating moving pictures, when performing coding and decoding processing to object data object is presented repeatedly and periodically, which means that reuse of the data is highly effective. Furthermore, in such games, continued presentation of object and deletion of object are determined depending on events which occur corresponding to user's operation, and thus a control is desired which permits reuse of data when continuing the presentation is determined.
Thus, in the buffer control of the prior art decoding apparatus, holding data after the processing at subsequent stage is not considered. Further, although it is unreal to hold the whole data stored in buffers of finite capacities, it is not possible to judge which object should be reused as well as to specify a storage period, which prevents improvement of processing efficiency and effective utilization of device resource by reuse of data stored in the buffer.